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Abstract 

Taking  a  subset  of  the  constraints  of  a  general  mixed  Integer  linear 
program  up  Into  the  objective  function  In  a  Lagrangean  fashion  (with  fixed 
multipliers)  yields  what  we  call  a  "Lagrangean  relaxation"  of  the  original 
program.  This  paper  gives  a  reasonably  comprehensive  development  of  the 
use  of  this  simple  device  In  the  context  of  branch-and-bound.  The  selective 
application  of  these  Ideas  can  yield  significant  Improvements  in  perfor¬ 
mance  for  special  classes  of  problems. 
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1.  INTRODUCTION 


The  general  Integer  linear  programming  problem  can  be 
written  as 

(P)  Minimize  cx  subject  to  Ax  ^  b 

x^°  Bx  £  d 

Xj  integer,  j  €  I  , 

where  b,  c  and  d  are  vectors  and  A  and  B  are  matrices 
of  conformable  dimensions,  and  the  set  I  consists  of  the 
subscripts  of  the  variables  required  to  be  integer.  Hie 
reason  for  distinguishing  two  groups  of  constraints  is  that 
the  second  of  these,  Bx  £  d  ,  is  supposed  to  have  special 
structure.  We  define  the  Lagrange an  relaxation  of  (P) 
relative  to  Ax  ^  b  and  a  conformable  nonnegative  Lagrangean 
vector  X  to  be i 


Minimize 

x>0 


cx+X  (b-Ax)  subject  to  Bx  ^  d 

Xj  integer,  j  c  I 


Hie  fruitful  application  of  ( PR in  specific  cases  requires 
judicious  partitioning  of  the  constraints  into  the  two  groups 
Ax  ^  b  and  Bx  ^  d  ,  and  an  appropriate  choice  of  X  £  0  . 
Three  specific  types  of  Bx  £  d  constraints  of  considerable 
practical  importance  are  introduced  in  Sec.  1.1  and  carried 
throughout  the  paper. 

Hie  purpose  of  this  paper  is  to  explore  the  usefulness 
of  (PR^)  a®  an  adjunct  to  branch-and-bound  or  implicit 
enumeration  methods  for  (P)  .  Hie  development  is  intended 
for  use  at  two  levels.  Pedagogically,  it  strives  for  a  sim¬ 
plified  and  unified  exposition  of  a  number  of  old  and  new 
developments  in  integer  programming.  As  a  research  effort 
it  aims  to  develop  what  appears  to  be  a  potent  general  ap¬ 
proach  to  the  design  of  improved  algorithms  for  special 
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classes  of  integer  programs.  Although  the  algorithmic  con¬ 
text  of  this  paper  is  the  branch-and-bound  approach  to 
integer  linear  programs,  it  should  become  obvious  to  the 
reader  how  these  ideas  can  also  be  applied  to  other  classes 
of  algorithms  and  problems.  For  instance,  a  promising  class 
of  cutting-planes  based  on  Lagrangean  relaxation  ideas  is 
obtained  as  a  part  of  our  development. 

No  claim  is  made  for  the  originality  of  the  notion  of 
Lagrangean  relaxation.  To  the  contrary,  it  has  been  used 
recently  by  Held  and  Karp  [16]  [17]  in  their  highly  success¬ 
ful  work  on  the  traveling-salesman  problem ;  by  Fisher  (  5  ] 
in  his  promising  algorithm  for  scheduling  in  the  presence  of 
resource  constraints;  by  Fisher  and  Schrage  [  6  ]  in  their 
proposed  algorithm  for  scheduling  hospital  admissions;  and 
by  Shapiro  [18]  and  Fisher  and  Shapiro  [  7  ]  in  the  context 
of  a  group  theoretic  approach  to  general  integer  programming. 

No  doubt  other  authors  have  also  made  special  application  of 
Lagrangean  relaxation  ideas  implicitly  if  not  explicity  in 
their  work.  We  should  also  mention  the  general  relevance  of  the 
literature  on  Lagrangean  methods  for  non-convex  optimization 
(e.g..  Brooks  and  Geoff rion  [2]  and  Greenberg  and  Robbins  [ 14] )  . 

The  general  plan  of  this  paper  is  as  follows.  The 
basic  results  concerning  the  relation  between  (P)  and 
(PR^)  are  collected  in  Sec.  2.  Duality  turns  out  to  play 
a  surprisingly  major  role.  In  Sec.  3  a  generic  LP-based 
enumerative  approach  for  (P)  is  reviewed,  and  the  four 
basic  uses  of  Lagrangean  relaxation  in  this  context  are  de¬ 
scribed.  The  basic  strategic  options  relating  to  the  use  of 
(PR^)  are  also  detailed.  Sec.  4  describes  how  cutting- 
planes  can  be  generated  based  on  (PR^)  •  The  "filter" 
constraint  of  Balas  [  1  ] ,  the  "profit  constraint"  of  Healy 
[15],  and  several  new  cuts  are  obtained  as  special  cases. 

Sec.  5  derives  the  penalties  of  Driebeek  [4  ]  and  Tomlin  [19] 
from  the  viewpoint  of  Lagrangean  relaxation,  and  new  improved 
penalties  are  developed  for  important  special  cases.  The 
use  of  penalties  in  strengthening  the  cuts  of  Sec.  4  and  de¬ 
riving  new  ones  is  also  explained.  In  Sec.  6  the  concept  of 


surrogate  constraints  as  developed  by  Glover  [13]  [14]  and 
Geoffrion  [  8  ]  is  shown  to  be  subsumed  by  the  Lagrangean 
relaxation  viewpoint.  Some  concluding  comments  are  given 
in  Sec.  7. 

Three  particular  but  important  cases  for  the  special 
constraints  Bx  £  d  are  carried  throughout  the  paper.  They 
serve  to  illustrate  general  ideas  in  a  concrete  way,  and 
also  to  emphasize  that  Lagrangean  relaxation  is  intended  to 
be  specialized  to  particular  problem  structures.  These  three 
cases  are  introduced  in  the  following  subsection.  The  final 
subsection  of  this  Introduction  summarizes  the  special  nota¬ 
tions  aid  assumptions  commonly  used  in  the  sequel. 

1.1  Three  Examples 

The  Lagrangean  relaxation  (PR^)  must  be  vastly  simpler 
to  solve  than  (P)  itself  in  order  for  it  to  yield  any  com¬ 
putational  advantage.  It  should  admit  a  closed  form  solution 
or  be  solvable  by  an  efficient  specialized 

algorithm.  Thus  the  constraints  Bx  ^  d  must  possess  con¬ 
siderable  special  structure.  Three  typical  examples  of  such 
structure  will  now  be  given.  These  examples  will  be  referred 
to  repeatedly  in  the  sequel. 

Example  1.  The  constraints  Bx  £  d  specify  only  upper  bounds 
on  some  or  all  of  the  variables.  For  instance,  in  0-1 
programming  problems  the  integer  variables  possess  upper 
bounds  of  unity.  It  is  easy  to  see  that  the  optimal  solution 
of  (PR^)  c*n  b®  written  down  by  inspection  of  the  signs  of 
the  collected  coefficient  vector  of  x  ,  namely  (c  -  XA)  . 

Example  2.  The  constraints  Bx  ^  d  are  as  in  Example  1  but 
also  include  some  generalized  upper  bounding  constraints  of 
the  form 
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(1)  2  x,  -  1  ,  k  -  1,2,...,  K  , 

J«4  1 

where  J^,  ...,JX  are  disjoint  subsets  of  I  •  Such  constraints 

perform  a  "multiple  choice"  function.  The  optimal  solution 
of  (PR^)  can  again  be  written  down  by  inspection,  with  a 
search  for  the  smallest  (c  -  \A)  j  now  being  necessary  over 

each  subset  . 

Example  3.  The  constraints  Bx  >  d  are  as  in  Example  1  but 
also  include  some  "switching"  constraints  of  the  form 

(2)  2  P^jXj  K  Pj^x^  *  1  ■*  •  •  •  » 

where  the  K  subsets  fk,J^}  are  disjoint,  x^.^Xg  are 

0-1  variables,  the  variables  in  are  continuous-valued, 

and  all  0  coefficients  are  strictly  positive.  This  type  of 
constraint  typically  arises  in  capital  expansion  models,  in 
the  familiar  capacitated  plant  location  problem,  for  example, 
x^  is  1  or  0  according  to  whether  or  not  a  plant  of  capa¬ 
city  is  built  at  the  kth  site,  for  jeJ^  corres¬ 

ponds  to  the  amounts  shipped  from  plant  site  k  to  various 
destinations,  and  the  p^'s  are  all  unity.  The  Lagrangean 
relaxation  (PR^)  can  still  be  solved  easily,  since  it  sepa¬ 
rates  into  K  independent  problems  of  the  form 

(3^)  Minimize  2  (c-XA)jXj  +  (c-XAj^x^ 

k 

subject  to  2  Pfcjxj  £  ^kk*k 
j€Jk 

0  £  Xj  <  U^jcJfc 

x^»0  or  1  , 


where  Uj  is  the  upper  bound  on  variable  x^  .  If  x^O  ,  it 

follows  from  the  positivity  of  that  x.*0  must  hold  for 

Kj  3 

all  jeJ^  .  If  x^*l  >  (3^)  becomes  a  trivial  linear  program 
sometimes  referred  to  as  a  "continuous  knapsack  problem"  with 
bounded  variables;  its  solution  is  easily  determined  by  com¬ 
parison  of  the  ratios 


(c-XA)../^  . 


The  best  of  the  solutions  obtained  under  tho  two  cases  x^O 

and  x^l  yields  the  true  optimal  solution  of  (3^)  .  From 

these  K  solutions  one  may  directly  assemble  the  optimal  solu¬ 
tion  of  (PR^)  • 


These  three  examples  serve  to  illustrate  some  of  the 
commonly  occuring  types  of  special  constraints  Bx  >  d  for 
which  the  associated  Lagrangean  relaxation  can  be  optimized  very 
efficiently.  In  most  practical  applications  of  integer  pro¬ 
gramming  there  are  several  obvious  and  tractible  choices  for  the 
constraints  to  be  designated  as  B  x  >  d  .  In  Held  and  Karp's 
excellent  work  on  the  traveling-salesman  problem  [16]  [17],  for 
example,  (PR^)  is  a  minimum  spanning  "1-tree"  problem  for 

which  highly  efficient  algorithms  are  available.  And  in  Fisher 
and  Schrage's  algorithm  for  hospital  admissions  scheduling  [6], 
(PR^ )  separates  into  a  relatively  simple  scheduling  problem 
for  each  patient. 


1.2  Notation  and  assumptions 

Notation  and  terminology  is  generally  standard  and  consis¬ 
tent  with  that  of  Geof frion  and  Marsten  [ 11] ,  a  survey  paper 
containing  additional  background  material.  However,  the  reader 
should  memorize  the  following  peculiar  notations:  if  («)  is 


6 


an  optimization  problem,  then  v(>)  is  its  optimal  value,  P ( •) 
is  its  set  of  feasible  solutions,  and  ("•")  refers  to  the  same 
problem  with  all  integrality  conditions  on  the  variables  dropped. 
The  vector  \  denotes  the  optimal  multiplier  vector  (dual  solu¬ 
tion)  associated  with  the  constraints  Ax  £  b  for  the  ordinary 
linear  program  (?)  . 

We  adopt  the  convention  that  the  optimal  value  of  an  in¬ 
feasible  optimization  problem  is  +  «  (resp.  -  ®)  if  it  is  a 
minimizing  (resp.  maximizing)  problem.  The  inner  product  of  two 
vectors,  be  they  row  or  column,  is  denoted  simply  by  their  justa- 
position. 

Two  benign  assumptions  are  made  throughout  this  paper  in 
the  interest  of  decluttering  the  exposition,  except  where  expli¬ 
citly  stated  to  the  contrary.  The  first  is  that  the  non-special 
constraints  Ax  £  b  are  all  inequality  constraints.  If  some  of 
these  constraints  were  given  as  equalities,  then  the  correspond¬ 
ing  components  of  X  would  not  be  required  to  be  nonnegative. 

This  is  the  only  change  required  to  accomodate  equality  con¬ 
straints.  The  second  assumption  is  that  the  special  constraints 
Bx  £  d  include  upper  bounds  on  all  variables.  This  obviates 
the  need  for  special  treatment  of  the  case  where  (p)  or  one  of 
its  relaxations  has  optimal  value  equal  to  -  •  , 
and  it  also  leads  to  some  notational  economies  as  explained  in 
Sec.  3.2.  This  assumption  is  consistent  with  the  vast  majority 
of  potential  applications.  It  is  a  simple  exercise  to  allow 
for  its  absence  in  the  all  of  the  results  to  follow* . 
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2 .  THEORY  OF  LAGRANGE  AN  RELAXATION 

The  term  relaxation  is  used  in  this  paper  in  the  following 
sense:  a  minimizing  problem  (Q)  is  said  to  be  a  relaxation 

of  a  minimizing  problem  (P)  if  F(Q)=F(P)  and  the  objective 
function  of  (Q)  nowhere  exceeds  in  value  that  of  (P)  on 
F  (P)  .  Clearly  (PR^)  is  a  relaxation  in  this  sense  for  all 

X  £  0  ,  for  the  extra  Lagrangean  term  X(b  -  Ax)  in  the 

objective  function  of  (PR^)  must  be  non-positive  when 

Ax  ^  b  is  satisfied.  Notice  that  the  common  practice  of 
relaxation  by  simply  throwing  away  some  of  the  constraints  is 
equivalent  to  Lagrangean  relaxation  with  X  *  0  .  Permitting 
X  f  0  allows  the  relaxation  to  be  tighter.  Notice  also  that 
if  an  optimal  solution  of  (PR^)  is  feasible  in  (P)  and 

satisfies  X  (b  -  Ax)  -  0  ,  then  it  must  be  optimal  in  (P)  . 

The  potential  usefulness  of  any  relaxation  of  (P)  , 
and  of  a  Lagrangean  relaxation  in  particular,  is  largely 
determined  by  how  near  its  optimal  value  is  to  that  of  (F)  . 
This  furnishes  a  criterion  by  which  to  measure  the  "quality" 
of  a  particular  choice  for  X  •  The  ideal  choice  would  be 
to  take  X  as  an  optimal  solution  of  the  concave  program 

(D)  Maximize  v(PR. )  , 

X£0  * 

which  we  have  designated  by  (D)  because  it  coincides  with 
the  formal  dual  of  (P)  with  respect  to  the  constraints 
Ax  £  b  (see,  eg.,  [  9]).  This  problem  in  turn  is  intimately 
linked  to  the  following  relaxation  of  (P)  : 

(P*)  Minimize  cx  subject  to  Ax  £  b 

x  x  cCofx^O:  Bx  2  d  and 

Xj  integer,  j  €  I)  , 
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where  Co  denotes  the  convex  hull  of  a  set.  It  may  be  diffi¬ 
cult  to  express  the  convex  hull  in  (P*)  as  an  explicit  set 
of  linear  constraints,  but  in  principle  this  is  always  possible 
and  so  (P*)  may  be  regarded  as  a  linear  program.  An  optimal 
multiplier  vector  corresponding  to  Ax  ^  b  will  be  denoted 
by  x*  when  (P*)  has  finite  optimal  value. 

Theorem  1  collects  the  important  relationships  between 
(PRX)  ,  (P)  ,  (P*)  ,  (P)  and  (D)  . 

Theorem  1. 


A.  P(P)^P(P*)3P(P)  ,  F(PRX)J(P) 

v(P)£v(F*)£v(P)  ,  v(PRx)£v(P)  for  all  X  £  0  . 

B.  If  (P)  is  feasible,  then  v(P)  £  v(PR^) 

C.  If  (r  ’)  is  feasible,  then 

v (D)  &.  Max  v  (PR  )  ■  v  (PR.  *)  ■  v(P*)  . 

x^o  >  X 

D.  Let  (P)  be  feasible  and  (PRX>  have  the 

following  Integrality  Property  for  all  x  £  0  t 

its  optimal  value  is  not  altered  by  dropping  the 
integrality  conditions  on  the  variables,  i.e., 
v(PRx)  m  v(PR^)  for  all  X^O  .  Then  (P*)  is 

feasible  and 

v(P)  -  v (PR—)  -  v(D)  -  v(PRx*)  ■  v(P*)  . 


I 


9 


Proof.  Part  A  is 

trivial.  Let  (P)  be  feasible. 

( 

Then 

v(P)  ■  Max  v  (PR. ) 

X£0  * 

by  the  dual  theorem 
of  linear  programming^ 

l 

-  v(PR^ 

by  the  definition  of  X 

£  v(PR-)  because  F (PR-Jo  F (PR-) 

< 

This  proves  Part  B.  An  identical  argument  (the  third  portion 
is  not  needed)  applied  to  (P*)  yields  the  conclusion  of 
Part  C  if  one  uses  the  following  observation  in  the  obvious 
ways 


i> 


V(PRX) 


Min  cx+X  (b-Ax) 
x 


subj.  to  x  e  Co(x£0:Bx£d 
and  Xj  integer,  j  e  I)  , 


which  holds  because  the  minimum  value  of  a  linear  function 
over  any  compact  set  is  not  changed  if  the  set  is  replaced 
by  its  convex  hull.  In  view  of  Parts  B  and  C,  to  prove 
Part  D  it  is  enough  to  show  v(P)  -  v(P*)  . 


±/  We  have  taken  here  the  '’partial"  dual  of  (P)  with 
respect  to  the  constraints  Ax  £  b  ,  rather  than  the  "full" 
dual  customarily  used  in  linear  programming.  See  Geoffrion 
(  9]  (especially  Sec.  6.1)  for  an  account  of  this  generali¬ 
zation  of  the  traditional  duality  theory.  It  is  easily 
verified  that  x  is  a  bona  fide  optimal  solution  of  the 
partial  dual  even  though  it  may  be  defined  in  terms  of  the 
full  dual. 
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v(P)  =  Max  v  (PR. )  by  duality 
\±Q  A 


*  Max  v  (PR. )  by  the  Integrality  Property 

J£0  x 


m 

■  Max  Min  cx+X  (b-Ax)  subj.  to  x  €  Co{x>0: 

5£0  x 

Bx£d  and  x^  integer,  j  €  1} 

by  the  observation  used  in  the 
proof  of  Part  C 

■  v(P*)  by  duality  . 

Notice  that  the  feasibility  of  (P*)  is  a  consequence  of 
the  fact  that  its  dual  has  finite  optimal  value. 

QED 

A  few  comments  on  the  significance  and  interpretation 
of  Theorem  1  are  in  order.  Part  A  simply  states  the  most 
obvious  relations  between  (P)  and  its  relaxations  (P)  , 
(P*)  and  (PR^)  •  Part  B  shows  that  x  ,  an  immediate 
byproduct  of  optimally  solving  the  standard  LP  relaxation 
(P)  ,  yields  a  Lagrangean  relaxation  that  is  at  least  as 
good  as  (P)  itself.  Thus  X  is  a  reasonably  good  feasible 
solution  of  (D) .  Part  D  shows  that  x  is  in  fact  the  opti¬ 
mal  choice  when  <PR^)  has  the  Integrality  Property  for  all 
X  *  0  .  in  this  case  Lagrangean  relaxation  can  do  no  better 
than  (P)  •  Part  C  is  of  interest  when  the  Integrality 
Property  does  not  hold.  It  shows  that  Lagrangean  relaxation 
can  do  as  well  as  (P*)  ,  which  in  some  cases  may  be  a  con¬ 
siderably  tighter  relaxation  than  (P)  --especially  when  the 


11 


Ax  ^  b  constraints  are  not  very  restrictive  by  comparison 
with  Bx  ^  d  .  In  the  extreme  case  that  Ax  ^  b  is  vacuous, 
evidently  v(P*)  »  v(P)  . 

The  Integrality  Property  clearly  holds  for  Example  1 
and  2  above  (one  may  assume  without  loss  of  generality  that 
the  upper  bounds  on  the  integer  variables  are  integers),  but 
it  does  not  hold  for  Example  3.  The  presence  or  absence  of 
the  Integrality  property  is  evident  in  many  applications  upon 
inspection  of  (PR^)  In  light  of  the  special  structure  of 
the  constraints  Bx  £  d  .  In  other  applications  one  may  be 
able  to  appeal  to  the  total  unimodularity  characterization 
of  natural  integer  solutions  of  linear  programming  problems 
(e.g.,  Veinott  and  Oantzig  [21]). 
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3.  THE  USE  OF  LAG  RANGE  AN  RELAXATION  WITH 
AN  LP-BASED  ENUMERATIVE  ALGORITHM 

Lagrangean  relaxation  can  be  viewed  ae  an  auxiliary 
device  useful  In  connection  with  an  LP-based  algorithm  for 
(P)  .  It  can  be  used  with  non-LP-based  algorithms  as  well, 
but  its  use  is  particularly  easy  and  natural  when  the  primary 
algorithm  for  (P)  solves  a  sequence  of  linear  programs. 

Since  virtually  all  currently  successful  general  integer 
linear  programming  algorithms  are  enumerative  as  well  as 
LP-based  (see  Sec.  3.1  of  [  11  ]),  we  choose  to  describe  the 
applications  of  Lagrangean  relaxation  in  this  particular 
context. 

3.1  Generic  LF-Based  Enumerative  Algorithm  for  (P) 

It  is  necessary  at  this  point  to  very  briefly  describe 
a  generic  LP-based  enumerative  algorithm  for  (P)  .  The  version 
given  here  is  not  the  most  general  possible,  but  it  does  compre¬ 
hend  the  current  generation  of  successful  implementations.  The 
description  is  based  on  portions  of  [  11  ] ,  which  can  be  con¬ 
sulted  for  further  details. 

The  unenumerated  portion  of  problem  (P)  is  represented 
at  any  given  time  by  a  list  of  so-called  candidate  problems, 
each  of  which  is  simply  (P)  with  certain  additional  con¬ 
straints  appended.  Each  of  these  constraints  stipulates 
that  the  value  of  one  of  the  integer  variables  must  lie  in 
a  certain  closed  interval  (degenerate  "point"  intervals  are 
allowed) .  When  a  candidate  problem  (CP)  is  examined  and 
it  is  determined  that  it  could  not  yield  a  feasible  solution 
to  (P)  which  is  better  than  the  best  feasible  solution 
previously  found,  then  (CP)  is  said  to  be  fathomed.  The 
best  currently  known  feasible  solution  of  (P)  is  called 
the  incumbent.  Its  cost  is  designated  by  z  .  When  a  can¬ 
didate  problem  is  examined  but  not  fathomed,  it  is  separated 


into  two  simpler  candidate  problems  (sometimes  called 
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subproblems)  by  the  addition  of  dichotomous  interval  con¬ 
straints  on  one  of  the  integer  variables.  The  variable 
selected  for  this  purpose  is  called  the  separation  variable. 
For  instance,  if  were  the  separation  variable  then  one 
of  the  subproblems  might  receive  the  new  constraint  x3  *  0  , 
while  the  other  would  receive  the  new  constraint  x3  ^  1  . 
(There  are  many  other  ways  to  separate  a  candidate  problem, 
but  this  type  of  separation  is  by  far  the  most  common.) 

An  outline  of  a  generic  LP-based  enumeratave  algorithm 
for  (P)  can  now  be  given. 


Step  1.  Initialize  the  list  of  candidate  problems 
to  consist  of  (P)  alone  and  set  z*  to  an 
arbitrarily  large  number. 

Step  2.  Stop  if  the  list  of  candidate  problems  is 
empty:  if  there  is  an  incumbent  then  it  must 
be  optimal,  otherwise  (P)  has  no  feasible 
solution. 

Step  3.  Choose  and  extract  one  member  of  the  candi¬ 
date  problem  list  to  become  the  current  candi¬ 
date  problem  (CP)  . 


Step  4.  Define  the  current  relaxed  candidate  problem 
(CPR)  to  be  (CP)  [  (CP)  with  the  conditions 

"Xj  integer,  j  €  I”  dropped]. 

Step  5.  Solve  (CPR)  by  linear  programming  or  some 
other  suitable  algorithm. 

Step  6.  If  (CPR)  proves  to  be  infeasible,  go  to 
Step  2. 
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Step  7.  If  the  optimal  value  of  (CP  )  proves  to 

I\ 

be  no  less  than  z*  ,  go  to  Step  2. 

D 

Step  8.  If  an  optimal  solution  x  of  (CPR) 
happens  to  be  optimal  in  (CP)  [i.e., 

xR  e  F  (CP)  and  v  (CPR)  =  cxR] ,  then  record 

this  solution  as  the  new  incumbent,  put 

D 

z*  ■  cx  ,  and  go  to  Step  2. 

Step  9.  Decide  whether  or  not  to  persist  in  attempt¬ 
ing  to  fathom  (CP)  .  If  so,  go  to  Step  10; 
otherwise  go  to  Step  11. 

Step  10.  Replace  (CPR)  by  another  (hopefully 

tighter)  relaxation  of  (CP)  ,  and  return  to 
Step  5. 

Step  11.  Select  a  separation  variable  j0,jQ  *'  I  , 

whose  optimal  value  x.  in  (CP)  was  not 

3o 

integral.  Separate  (CP)  into  two  subproblems 
via  the  dichotomous  interval  constraints 
"xi  1  [5,  ]"  and  "x.  2  [x,  ]  +  1"  , 

where  [x-  ]  stands  for  the  integer  part  of 
3o 

x  •  ,  and  add  the  two  subproblems  to  the 

3o 

candidate  list. 

By  appropriate  specializations  of  Steps  3,  9,  10  and  11, 
the  above  serves  to  describe  MPSX-MIP,  OP  HE  LIE  MIXED,  the 
mixed  integer  option  of  UMPIRE,  and  integer  programming  codes 
developed  by  Beale-Small,  Dakin,  Davis,  Davis-Kendrick- 
Weitzman,  the  author,  Mitra,  and  others.  See  Sec.  3.1  of 
[  11]  for  references. 
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3.2  preliminary  Remarks  on  the  use  of  Lagrangean  Relaxation 


The  notion  of  relaxation  played  a  central  role  in  the 
generic  algorithm  just  reviewed.  At  Step  4  the  integrality  con¬ 
ditions  on  the  current  candidate  problem  are  dropped,  making  it 
possible  to  compute  an  optimal  solution  of  the  relaxed  candidate 
problem  using  the  efficient  methods  of  linear  programming.  This 
results  in  fathoming  (at  Step  6  or  7  or  8) ,  or  in  guidance  con¬ 
cerning  how  to  separate  the  candidate  problem  into  easier  sub- 
problems  (at  Step  11)  .  The  whole  point  is  to  gain  access  to  the 
powerful  machinery  of  linear  programming. 

Lagrangean  relaxation  is  useful  as  a  means  of  gaining 
access  to  other  strong  but  still  easily  solved  relaxations  of 
the  candidate  problem.  These  relaxations  can  take  advantage  of 
the  LP  solutions  generated  at  Step  5  in  a  natural  way,  and  can 
be  designed  to  exploit  many  common  special  structural  features 
found  in  practical  integer  programming  applications. 

There  are  four  basic  uses  of  Lagrangean  relaxation  within 
the  context  of  the  generic  algorithm:  as  a  means  of  fathoming 
the  current  candidate  problem,  of  guiding  its  separation  into 
more  tractable  subproblems,  of  restricting  the  range  of  some  of 
the  variables  prior  to  separation  (without  sacrificing  the  opti¬ 
mal  solution),  and  of  generating  improved  feasible  solutions  of 
(P)  .  Each  of  these  uses  will  be  discussed  in  turn.  An  optional 
fifth  use  pertaining  to  the  construction  of  cutting-planes  (cf. 
Step  10)  is  treated  separately  in  Sec.  4. 

Applications  of  Lagrangean  relaxation  within  the  context  of 
the  generic  algorithm  require  it  to  be  applied  to  the  candidate 
problems  rather  than  to  (P)  itself.  Our  assumption  that 


range  restrictions  on  all  variables  are  incorporated  into 
the  special  constraints  Bx  £  d  implies,  however,  that 


This 


obviates  the  need  to  introduce  special  notation  for  the 


additional  restrictions  associated  with  (CP)  •  We  may  thus  use 
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exactly  the  same  notation  for  (PR^)  and  (CPR^),  the 
Lagrangean  relaxation  of  (CP),  and  the  results  of  Sec.  2  apply 
to  (CP)  as  well  as  to  (P)  .  Separation  constraints  more  com¬ 
plex  than  the  simple  interval  constraints  we  have  assumed  at 
Step  11  would  require  a  more  scrupulous  notation. 

The  reader  will  notice  in  what  follows  that  X  is  typically 


taken  to  be  X — a  byproduct  of  the  linear  programming  solution 
to  (CP)  .  This  is  a  natural  choice,  but  other  possible  strate¬ 
gies  for  selecting  X  are  discussed  in  Sec.  3.7.  Taking 
also  implies  that  Lagrangean  relaxation  is  invoked  after  (CP) 
has  been  solved  as  a  linear  program,  presumably  as  a  result  of 
exercising  the  option  of  Step  10.  Secs.  3. 3-3. 6  are 


written  under  this  presumption  for  ease  of  exposition.  But  Sec. 

3.7  discusses  the  frequently  attractive  alternative  of  invoking 
Lagrangean  relaxation  before  (CP)  is  solved,  using  a  previously 
computed  X  . 

3.3  Fathoming 

Suppose  that  (CP)  could  not  be  fathomed  via  the  standard 
LP  relaxation  (CP)  .  Then  at  Step  9  one  could  decide  to  persist 
in  the  attempt  to  fathom  (CP)  by  setting  up  the  Lagrangean  re¬ 
laxation  (CPF^-  )  at  Step  10.  Upon  return  to  Step  5,  (CPRj^  ) 
would  be  solved  by  some  special  method  which  exploits  the  struc¬ 
ture  of  the  special  constraints.  Step  6  would  check  for  fathoming 
by  infeasibility:  F(CPR^  )  *j 1  .  Step  7  would  check  for  fathoming 
by  value:  v(CPR^-  )  z*  .  And  Step  8  would  check  for  fathoming 
by  the  detection  of  an  optimal  solution  of  (CP)  :  the  optimal 
solution  of  (CPF^-  ) ,  say  x  ,  would  be  optimal  in  (CP)  if  it 
is  feasible  in  (CP)  and  \  (b-AxR)  *  0  .  j 

Theorem  l.D  sheds  some  light  on  the  probable  fathoming 
effectiveness  of  (CPR^-  ^  wtien  used  as  described  as  a  supplement 
to  the  standard  relaxation  (CP)  .  It  reveals  that  the  tests  for 
fathoming  by  infeasibility  (Step  6)  or  by  value  (Step  7)  cannot 

)  if  (CPR^  )  has  the 


be  effective  when  applied  to  (CPR^- 
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Integrality  Property  for  all  X£0  (as  is  the  case,  it  will  be 
recalled,  for  Examples  1  and  2).  This  is  because  v(CPR^)  must 
equal  v(CP).  Step  8,  on  the  other  hand,  can  succeed  for  (CPR^) 
even  though  it  fails  for  (CP) . 


3.4  Guiding  separation 

Separation  of  (CP)  is  necessary  when  the  attempts  to 
fathom  it  have  failed.  The  Lagrangean  relaxation  (CPR^-)  can 
be  useful  at  Step  11  in  selecting  a  separation  variable 
and  in  predicting  which  of  the  two  resulting  subproblems  is 
more  likely  to  contain  the  better  feasible  solution  of  (P) . 
This  involves  computing  the  conditional  bounds 


A  _ 

Vj)  -  v(CPR^|Xj  <  [Xj]) 

Vy(j)  =  vCCPft^Xj  a  IXj]  +  1) 


for  all  indices  j  in  If,  defined  to  be  the  subset  of  I  cor¬ 
responding  to  variables  that  are  fractional  in  the  LP  solution 
x  of  (CP).  The  "I”  notation  indicates  that  the  constraint 
following  it  (a  simple  range  restriction  on  a  single  variable) 
is  appended  to  (CPR^-) .  Clearly  (CPK£-|xj  <  [XjD  is  a  legiti¬ 
mate  Lagrangean  relaxation  of  (CP[xj  £  [Xj]),  and  similarly  for 
(CPR^|Xj  ;>  [Xj]  +  1).  Thus  the  numbers  VD(j),  and  V^(j) 
respectively  represent  conditional  lower  bounds  on  the  optimal 
values  of  the  new  "downward"  and  "upward"  subproblems  which 
would  result  if  (CP)  were  separated  on  the  variable. 

As  an  example  of  the  use  of  conditional  bounds,  the  sep¬ 
aration  variable  could  be  chosen  so  as  to  maximize  the  larger 
of  Vp ( j)  and  Vpfj)  over  all  eligible  j.  Several  successful 
integer  programming  codes  have  employed  an  analagous  criterion 
applied  to  LP-based  "penalties"  —  a  concept  which,  as  we 
shall  see  in  Section  5,  is  intimately  related  to  (4).  Once 

a  separation  variable  j  is  selected,  the  smaller  of  V. (j  ) 

o  D  o 

and  Vy  (j^)  indicates  which  of  the  two  resulting  subproblems 
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appears  likely  to  contain  the  best  feasible  solution  of  (P) . 
This  information  is  useful  for  deciding  which  of  the  two  sub¬ 
problems  should  be  selected  from  the  candidate  list  first  at 
a  future  execution  of  Step  3. 

The  usefulness  of  (CPRr-)  as  a  guide  to  separation  depends 

A 

primarily  on  how  well  VD(j)  and  V^j)  correlate  with  the  true 
optimal  values  of  the  corresponding  candidate  subproblems. 

For  Examples  1  and  2,  unfortunately,  it  turns  out  that 


(5)  v(CP)  =  VD(j)  =  Vytj)  for  all  j  €  If 


if  (4)  is  interpreted  literally  (more  on  this  in  Sectica  5) . 
This  fact  and  the  general  situation  to  be  expected  when  the 
Integrality  Property  holds  are  the  subject  of  Theorem  2. 


(6) 


Theorem  2 


Let  (CPR^ )  have  the  Integrality  Property  for  all  \  £  0, 
and  iet.  (CP)  have  a  finite  optimal  value.  Then  for 
all  j  f  lf 


V  (CP) 


v(CPR^)  £ 


i  Vu(j) 


with  equality  holding  in 
at  least  one  of  the  two 
cases. 


In  the  special  situation  of  Examples  1  and  2 ,  equality 
holds  in  (6)  for  both  VD ( j )  and  Vy(j). 


Proof 

The  equality  in  (6)  is  a  direct  consequence  of  Theorem 
l.D.  The  second  relation  follows  from  the  fact  that  the  two 
conditions  MXj  £  [Xj]"  and  "Xj  ^  [Xj]  +  1"  are  mutually  exclu¬ 
sive  and  exhaustive  (since  x^  is  required  to  be  integer-valued) . 

The  strengthened  conclusion  for  Examples  1  and  2  follows 
from  the  fact  that  x  and  x  must  satisfy  the  optimality  conditions 
for  (CP)  written  so  as  to  introduce  a  multiplier  vector  only 


/ 


for  the  constraints  Ax  ^  b: 

(i)  x  minimizes  cx  +  X  (b-Ax)  subject  to  Bx  >  d 
x  ^  0 

(ii)  X  £  0 

(iii)  X  (b-Ax)  =  0 

(iv)  Ax  £  b. 

We  need  only  condition  (i) .  Consider  now  the  objective 

A 

coefficient  (c-XA)  ?  for  j  f  i^.  For  Example  1  this  coef¬ 
ficient  must  be  0,  for  otherwise  the  term  (c-XA)  ^x*  could 
be  reduced  by  letting  x^  depart  from  x^  in  the  opposite 
direction  from  the  sign  of  (c-XA)?.  The  same  is  true  for 

a  3  «. 

Example  2  if  j  is  not  a  member  of  any  set;  and  if  j  is  a 
member  of  some  then  there  must  be  a  tie  in  the  smallest 
objective  function  coefficient  amongst  the  free  variables 
in  the  same  subset  as  xj  ,for  otherwise  the  term  Z^c-Xa)^ 
(sum  over  the  free  variables  just  specified)  could  be  reduced 
by  increasing  (decreasing)  x^  if  (c-XA)^  is  (is  not)  the 
uniquely  smallest  coefficient.  The  conclusion  regarding 
(c-XA)j  in  Examples  1  and  2  imply  that  x^  can  be  forced  to 
either  [x^  ]  or  [x^]  +  1  without  disturbing  the  optimal  value 
of  (CPRy) .  The  desired  conclusion  (5)  now  follows.  Q.E.D. 

Thus  Lagrangean  relaxation  with  X  ■  X  appears  to  be 
useless  for  guiding  separation  in  Examples  1  and  2,  since  (5) 
gives  no  discrimination  whatever  among  possible  separation 
variables  or  between  the  resulting  subproblems.  The  general 
case  in  which  the  Integrality  Property  holds  is  perhaps  a  little 
more  promising,  but  at  least  one  of  the  bounds  VQ(j)  and  V^(j) 
is  stuck  at  v(CP)  for  each  j  e  I.  [we  remark  in  anticipation 
of  Section  5  that  the  situation  can  be  significantly  improved 
if  (4)  is  computed  using  an  alternative  representation  of  x^ 
in  terms  of  the  non-basic  variables  in  the  optimal  solution 
of  (CP). 3 
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Lagrangean  relaxation  is  much  more  promising  at  Step  11 
when  the  Integrality  Property  does  not  hold.  Then  both  VD ( j ) 
and  Vy(j)  may  exceed  v(CP)  and  may  thereby  provide  useful 
discrimination  among  possible  separation  variables  and  the 
resulting  candidate  subproblems.  Example  3  illustrates 
nicely  how  this  can  be,  and  is  worth  examining  in  some  detail. 
Suppose  for  simplicity  that  all  variables  are  in 
Consider  the  initial  candidate  problem  (CP)  *=  (P)  [an  analysis 
similar  to  the  one  belov  applies  to  any  subsequent  candidate 
problem].  Hie  optimal  value  of  (3^)  with  X  *  X  and  with  x^ 
fixed  is  denoted  by  v^(x^).  It  is  known  from  the  elementary 
theory  of  parametric  linear  programming  that  v^fx^)  i-8  piece- 
wise-linear  and  convex  as  a  function  of  x^.  A  graph  of  this 
function  might  look  as  in  Figure  1.  In  this  picture  v^fl) 
happens  to  be  greater  than  zero*  although  it  could  also  be 


Hypothetical  Graph  of  the  Function  v^f*) 


less  than  or  equal  to  zero.  Note  that  v^  (0)  necessarily 
eouals  zero,  as  is  evident  upon  inspection  of  (3^) .  The 
minimum  of  the  function  is  shown  as  occurring  at  x^.  This 
is  true  because  (x,X)  must  satisfy  the  optimality  conditions 
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for  (P)  stated  as  in  the  proof  of  Theorem  2.  Hie  separability 
of  these  conditions  implies  that  x^  and  x^,  j  €  Jk>  are  opti¬ 
mal  in  (3^)  with  X  *  7  and  with  the  integrality  requirement 
on  x^  dropped  —  and  hence  that 

^(x^)  -  Minimum  ^(x^) 


It  also  follows  from  the  optimality  conditions  that 

K 

(7)  v(P)  ■  Xb  +  Vfctefc)* 

From  the  introductory  discussion  of  Example  3  we  have 

K 

(8)  vtPRj)  «  Xb  +  Z^ml  v(3k)  # 

Clearly 

v(3Jc)  -  Min  -[o,  vk(l)}2vx^^ 

We  can  now  see  plainly  from  (7)  and  (8)  that  v(p)  £  v(PR^) 
and  that  any  difference  is  exactly  equal  to 

(9)  l"ln  ^°’ Vk  a) } -  v*  ^  1  • 

This  expression  has  a  simple  interpretation  in  terms  of 
Figure  1.  For  obvious  reasons  there  can  be  no  contribution 
to  this  difference  for  the  terms  such  that  x^  happens  to  be 
0  or  1,  but  for  k  such  that  is  fractional  it  is  very  likely 
that  there  will  be  a  contribution. 
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Thus,  we  see  why  v(P)  <v(PR^)  ia  likely  whenever  there 
are  fractional  integer  variables  in  the  LP  solution  of  (P) 
for  Example  3.  In  this  event  both  vQ(k)  and  vy(k)  obviously 
must  exceed  v(P).  The  magnitude  of  these  conditional  bounds 
are  easily  interpreted  in  terms  of  graphs  like  Figure  1.  For 
the  particular  case  of  Figure  1, 

vQ(k)  -  v(PR^) 

Vy(k)  -  v(PR^)  +  vk(l)  . 


3.5  Range  Restriction 

Sometimes  during  the  attempted  fathoming  or  separation 
of  a  candidate  problem  it  is  discovered  that  some  of  the 
range  restrictions  on  the  variables  can  be  reduced  without 
losing  an  optimal  solution  of  (P) .  The  range  restrictions 
may  then  be  tightened  and  the  algorithm  continues  as  before. 
With  zero-one  variables,  this  is  sometimes  known  as  making 
a  "forced  choice"  of  a  variable. 

Lagrangean  relaxation  can  assist  in  making  such  dis¬ 
coveries  when  the  conditional  bounds  of  (4)  are  computed. 

. . .  * 

If  Vp ( j)  ^  z  happens  to  hold,  then  the  lower  range  restriction 
on  x.  obviously  can  be  tightened  to  x^  ^  +  1.  Similarly, 

vy (j)  ^  *  implies  that  the  upper  range  restriction  on  x^  can 
be  tightened  to  Xj  £  fxj].  It  is  even  possible  that  both 
Vp ( j )  ;>  z*  and  Vy(j)  ^  z*  hold,  in  which  case  it  is  clear  the 
current  candidate  problem  is  fathomed. 

Similar  reasoning  can  be  applied  to  j  €  I  even  if  Xj  is  integer: 
v  (CPRyJx j  £  x j-1)  ^  z* 
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implies  that  2  xj  can  be  imposed,  and 
v(CPR^|Xj  ^  Xj  +  1)  ^  z* 

implies  that  x .  £  x.  can  be  imposed.  Note  that  either  of 

J  J  ,  v  * 

these  situations  can  occur  even  though  v(CPRr)  <  z  . 

A 

All  of  the  conclusions  reached  in  the  previous  section 
regarding  the  magnitudes  of  the  conditional  bounds  VQ(j) 
and  V^(j)  are  of  direct  interest  with  regard  to  range  re¬ 
striction. 

3.6  Generating  Improved  Feasible  Solutions 

Recall  from  Section  3.3  that  (CPRr-)  leads  to  the  fathom- 

*  R 

ing  of  (CP)  at  Step  8  if  the  optimal  solution  x  of  (CPRr)  is 

_  R  R  * 

feasible  in  (CP)  and  satisfies  X  (b-Ax  )  -  0.  If  x  is  feasible 
__  R 

in  (CP)  but  x  (b-Ax  )  ?  0,  then  it  is  worth  checking  whether 

R  *  D 

cx  <  z  .  If  so,  x  should  be  recorded  as  a  new  incumbent 
and  z  should  be  updated  even  though  (CP)  is  not  fathomed. 

D 

Step  8  can  be  broadened  still  further  if  x  is  not 
feasible  in  (CP) .  Adjusting  x  in  some  ad  hoc  way  so  as  to 
recover  feasibility  may  well  lead  to  an  improved  feasible 
solution  of  (P) .  This  is  exactly  the  same  tactic  as  is 
commonly  used  with  the  ordinary  LP  relaxation  (CP) ,  where  the 
LP  solution  is  rounded  to  satisfy  integrality  in  the  hope  that 
it  will  become  feasible  in  (P)  and  have  a  value  better  than 
that  of  the  incumbent. 

'Rie  effective  design  of  feasible  solution  generators  along 
these  lines  usually  must  be  tailored  to  the  specific  problem 
structure  at  hand. 

3.7  Strategic  actions  for  the  Use  of  Laaranaean  Relaxation 
The  exposition  of  the  preceding  subsections  presumes  for 

the  most  part  that  Lagrangean  relaxation  is  invoked  only  after 
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(CP)  is  solved,  and  that  x  is  always  taken  to  be  the  x  cor¬ 
responding  to  (CP) .  This  might  be  called  the  Post-LP  strat¬ 
egy  with  the  natural  choice  for  x.  It  is  important  to  recog¬ 
nize  that  there  is  also  what  might  be  called  a  pre-LP  strategy, 
and  that  the  candidate  problem  counterparts  of  (D)  and  (P  ) 
may  yield  better  choices  of  x  than  (CP)  when  the 

Integrality  Property  does  not  hold  for  (CPR^) . 

The  idea  of  the  pre-LP  strategy  is  to  invoke  (CPR^)  just 
befoce  (CP)  is  to  be  solved.  The  hope  is  that  (CP)  can  there¬ 
by  be  fathomed  without  having  to  resort  to  the  more  expensive 
linear  program  (CP) .  The  natural  choice  for  x  is  to  tne  a 
multiplier  vector  saved  from  a  previous  execution  of  Step  5 
at  which  the  standard  LP  relaxation  of  a  candidate  problem 
was  addressed.  The  x  used  should  preferably  be  the  one  cor¬ 
responding  to  the  prior  candidate  problem  most  closely  related 
to  the  current  one;  or,  if  this  proves  cumbersome,  simply 
the  last  one  generated.  Notice  that  it  is  particularly  worth 
saving  for  this  purpose  the  multiplier  vector  for  a  candidate 
problem  which  is  fathomed  by  value  via  its  LP  relaxation 
[v(CPR^)  ^  z  necessarily  holds  for  this  (CP)  and  its  associated 
X].  The  computational  experience  cited  in  Section  3.1.5  of 
[11]  (see  also  Geoffrion  [8])  shows  that  the  pre-LP  strategy 
can  be  quite  effective  even  in  the  context  of  Example  1. 

The  other  point  we  wish  to  elaborate  upon  is  that,  for 
both  the  pre-LP  and  post-LP  strategies,  it  is  possible  to  use 
a  x  other  than  an  optimal  multiplier  vector  associated  with 
the  usual  LP  relaxation  of  a  candidate  problem.  One  may  wish 
to  do  this,  for  instance,  when  (CP)  appears  to  be  excessively 
expensive  to  solve  for  a  truly  optimal  solution  but  techniques 
are  available  for  generating  a  reasonably  good  suboptimal 
multiplier  vector.  Fisher  [  5  ]  has  used  this  variant  quite 
successfully  in  the  context  of  scheduling  in  the  presence  of 
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resource  constraints.  Here  (CP)  has  such  a  large  number  of 
columns  that  it  is  impractical  to  even  write  down  explicitly 
much  less  solve  optimally.  Nevertheless,  he  was  able  to  work  out 

< 

a  means  of  implementing  what  is  essentially  the  dual  Simplex 
Method  for  finding  good  dual  solutions  at  reasonable  computational 
cost . 

(,  A  second  reason  for  using  X  other  than  \  is  evident  from 

Theorem  1,  which  shows  that  better  choices  than  X*  may  exist  when 
(CPR^)  does  not  have  the  Integrality  Property  for  all  X  >  0  . 

One  then  looks  to  the  candidate  problem  counterparts  of  (D)  or 
(P  )  for  superior  X's.  If  available,  X"  furnishes  a  good  start¬ 
ing  point  to  improve  upon.  One  possibility  is  to  apply  an  ascent 
technique  to  the  counterpart  of  (D) ,  which  is  a  concave  program, 
so  as  to  obtain  at  least  a  near-optimal  solution.  This  has  been 
C  done  by  Held  and  Karp  [17]  and  by  Fisher  and  Shapiro  [7]  .  Another 

possibility  is  to  apply  a  Dantzig-Wolfe  decomposition  technique  to 

it 

the  counterpart  of  (P  )  ,  with  the  convex  hull  therein  represented 
in  terms  of  its  extreme  points  (cf.  Section  3  of  Held  and  Karp 
l  [16]  and  Brooks  and  Geoff  rion  [2]).  The  column -generation  problems 

would  be  precisely  of  the  form  (PR^)  •  The  calculations  would  very 
likely  be  terminated  before  complete  optimization,  due  to  the 
characteristically  slow  asymptotic  convergence  of  this  technique. 

I  Other  specialized  techniques,  both  exact/heuristic,  can  be  devised 

for  (O)  or  (P  )  in  particular  applications.  We  feel  that  this 
is  likely  to  be  a  fruitful  area  of  investigation  when  engineering 
a  computer  code  for  (P)  . 
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4.  CUTS  BASED  ON  LAG  RANGE  AN  RELAXATION 


For  present  purposes  a  cut  is  any  linear  inequality 
defined  relative  to  a  candidate  problem  which  must  be  satis¬ 
fied  by  all  of  its  feasible  solutions.  Generally  we  seek 
cuts  which,  when  appended  to  the  standard  LP  relaxation  (CP) 
of  a  candidate  problem,  cause  the  optimal  value  of  (CP)  to 
increase  as  much  as  possible.  It  is  therefore  desirable  for 
a  cut  to  have  the  property  that  it  is  not  satisfied  by  the 
available  optimal  solution  to  (CP)  .  Cuts  are  one  of  the 
most  important  ways  of  carrying  out  Step  10  of  the  generic 
enumerative  approach  described  earlier,  that  is,  of  tighten¬ 
ing  an  LP  relaxation  of  an  integer  program.  We  shall  see 
that  the  notion  of  Lagrangean  relaxation  leads  quite  directly 
to  a  remarkably  flexible  family  of  cuts. 

To  be  specific  we  shall  consider  the  initial  candidate 
problem  only,  namely  (P)  itself.  It  is  a  simple  matter  to 
see  how  the  ideas  developed  below  can  be  applied  to  any  candi¬ 
date  problem. 


4.1  Introduction  to  Lagrangean  Cuts 

The  fundamental  idea  for  generating  cuts  based  on  (PR^) 

for  any  specific  \  £  0  is  this.  Select  a  subset  of  the 

variables  of  (P)  ,  say  (x.,...,xn  )  ,  for  which  it  is 

l  nl 

possible  to  determine  an  explicit  linear  function 
4(x^,  ...,x^  )  satisfying 

(10)  |(x1,...,x|lJ  -  vlPRjx^...,*^) 


for  all  vectors  (x^...^  )  which  are  part  of  a  eoaie  feasible 

solution  of  (PR.)  .  Here  v(PR  |x.,...,xl;  )  is  intended 

A  A  1 

as  a  functional  notation  referring  to  the  optimal  value  of 
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(PR  )  with  the  additional  restriction  that  the  specified 

A 

variables  take  on  the  specified  values.  (If  a  particular 

vector  (x.,...,x  )  cannot  be  completed  so  as  to  yield  a 

i  n1 

feasible  solution  of  (PR.)  ,  then  by  convention 

A 

v (PR,  |x......x  )  is  defined  to  be  +  «  at  this  point.) 

K  1  nl 

The  cut  deriving  from  (10)  is 


(ID 


i(x1,  ...,xn^)  £  cx  . 


This  is  a  legitimate  cut  because 


£(xlf  ...,x  )  ■  v(PRx|xx,  ...,xn^)  <1  v(p|x1,...,xn^)  £ 


cx 


holds  for  all  x  feasible  in  (P)  . 

At  least  two  choices  for  (x.,...,x  )  are  always  per- 

1  nl 

missible  regardless  of  the  particular  structure  of 
(PR^)  s  the  empty  set  and  the  full  set.  For  the  empty  set 

the  function  t  is  just  the  constant  v(PR  )  .  Thus  (11) 

A 

becomes  simply 

(12)  v(PR^)  £  cx  . 

Note  that  (12)  cannot  be  violated  by  x  ,  an  optimal  solution 
of  (P)  ,  when  (PR^)  has  the  Integrality  Property  for  all 

X  £  0  .  In  that  case  v(PR^)  £  v(p)  ■  cx  must  hold  by 

Theorem  l.D,  and  so  appending  (12)  to  (P)  cannot  increase 
its  optimal  value.  Theorem  l.D  also  reveals  that  \  is  the 
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best  choice  of  x  in  this  case,  since  (12)  roust  then  hold 
with  exact  equality  at  x  .  In  the  absence  of  the  Integral¬ 
ity  Property,  (12)  will  be  violated  at  x  if  v(P)<v(P*) 
and  x  is  sufficiently  near  optimal  in  (D) . 

Now  consider  the  choice  where  (x.,...,x  )  consists 

i  n1 

of  all  of  the  variables.  Then 

v(PRjJx)  =  cx  +  X  (b  -  Ax) 

for  all  x  feasible  in  (PR.)  and  (11)  becomes 

A 

(13)  cx  +  X  (b-Ax)  ^  cx  or  X  (b-Ax)  £  0  . 

This  cut  obviously  cannot  be  violated  by  x  or  by  any  other 
feasible  solution  of  (P)  ,  since  then  b  -  Ax  <£  0  .  Thus 
(13)  is  ineffective  as  a  means  of  tightening  the  relaxation 
(P)  .  We  remark  in  passing  that  (13)  with  X  equal  to  X 
is  precisely  the  " filter"  constraint  proposed  by  Balas  [  1  ] 
in  the  context  of  Example  1  (as  noted  in  the  proof  of 
Theorem  2,  X (b-Ax)  «  0  ). 

The  particular  variables  which  play  the  role  of 

(x.,...,xn  }  will  be  called  the  cut  variables,  and  of  course 
1  nl 

they  need  not  be  the  first  nj_  consecutive  components  of 
x  .  other  choices  besides  the  empty  set  and  the  full  set 
are  usually  permissible  in  that  they  meet  the  requirement  of 
(10)  that  v(PR^  Ix^, .  ..,xn  )  must  be  linear  over  the  set  of 

vectors  (x1,...,x  )  which  are  part  of  a  feasible  solution 

A  nl 

of  (PR^)  •  Among  the  permissible  choices,  one  seeks  to 
maximise  the  depth  of  the  resulting  cut  in  some  sense.  Since 
we  have  in  mind  using  (11)  at  Step  10  of  the  generic 
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enumerative  algorithm,  one  appealing  criterion  for  measuring 
"depth"  is  the  amount  of  violation  of  (11)  evaluated  at  x  : 

(14)  l(x,,...,x  )  -  cx  . 

i  nl 

If  there  are  several  permissible  choices  of  cut  variables 
which  maximize  the  cut  depth  then  it  seems  reasonable  to 
turn  to  a  secondary  criterion  of  having  as  many  cut  variables 
as  possible  since  this  tends  to  strengthen  the  cut  when  evalu¬ 
ated  at  any  feasible  solution  of  (PR^)  *  This  is  evident 

from  the  obvious  inequality 

£V<PRx|xl,...,Xni,Xni+1>  . 

The  other  principal  question  arising  in  applications 
is  the  choice  of  X  .  The  cut  depth  criterion  (14)  serves 
as  well  to  discriminate  among  alternative  choices  of  X  as 
it  does  among  alternative  choices  of  cut  variables.  Hie 
most  conspicuous  choice  is  certainly  X  ,  though  one  must  . 
allow  that  another  choice  may  prove  superior  —  especially 
when  the  Integrality  Property  does  not  hold. 

Let  us  consider  now  our  three  examples  with  the  choice 
X  -  X  . 


4.2  Cuts  for  Example  1 

For  example  1  it  turns  out  that  any  subset  of  cut  vari¬ 
ables  is  permissible.  This  is  due  to  the  fact  that  (PR^) 

separates  into  as  many  independent  univariate  subproblems 
as  there  are  components  in  x  .  Evidently 


(15)  vfPR-lx^.. 


•*n  > 


V(PRX 


Iv 

*  '*1 


'...■X  -0)+Z  (c-XA)  jXj 

ni  j-i  j  j 
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for  every  )  which  is  part  of  a  feasible  solution 

i  nx 

to  (PR-)  .  Thus  (11)  becomes 

A 

ni 

(16)  v(PR-|x."..  .«x„  *0)  +  2  (c-XA) 4x .  £  cx  . 

a •  *  j  j 

The  depth  of  this  cut  at  x  is  zero  because 

ni-- 

v (PR-  |x.=. .  .=x„  =0)+ 2  (c-XA)  jX .  ■  v(PR-)  ■  cx 
A  *  n2.  jsl  J  J  * 

by  Theorem  l.D.  Since  all  cuts  have  the  same  depth  by  cri¬ 
terion  (14) ,  the  proper  choice  is  to  take  (x.,...,x.  }  to 

l  nx 

be  the  full  set.  Then  (16)  coincides  with  (13)  with  X*r  . 

4.3  Cut 8  for  Example  2 

For  Example  2  it  also  turns  out  that  any  choice  of  cut 
variables  is  permissible,  and  that  all  of  the  resulting  cuts 
have  exactly  zero  depth  at  x  .  We  omit  the  straightforward 
but  somewhat  tedious  verification  of  this  result.  Thus  (13) 
with  X-X"  is  again  the  best  cut. 

4.4  Cuts  for  Example  3 

The  situation  for  Example  3  is  a  little  more  complex.  Re¬ 
call  that  (PR^-)  separates  into  K  independent  subproblems  of 
the  form  (3^)  and  into  as  many  additional  univariate  subproblems 
as  there  are  variables  which  do  not  appear  in  any  of  the  subpro¬ 
blems  of  type  (3^).  Denote  the  subscripts  of  these  last  vari¬ 
ables  by  T.  Then 

-  K 

v (PRr)  -  Xb  +  2  v (3.J 
x  k-1  * 

Min  (c-XA)^Xj  subj.  to  Q^x^u^ 

and  Xj  integer  if  jel 


+  2 
jeT 
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Consider  selecting  for  the  cut  variables  any  subset  of  the 
0-1  switching  variables,  say  where  1  i  K  , 

plus  any  subset  S  of  T  . 

Evidently 

(17)  vfPR-jx^ .  ..,xK^?x^  for  j  €  S)  ■ 

and  Xj*0  for  jcS) 

+  2  M&JXb)  +  2  (c-XA).x. 
k«l  *  K  j€S  33 

holds  whenever  x^  is  0  or  1  for  l£k£K^  and  O^x^u^ 

for  j  €  S  and  also  x^  is  integer  for  j  €  IOS  .  The 
binary  nature  of  x^  makes  it  possible  to  write  down  a 
linear  function  of  x^  which  coincides  with  vt3jc!*jc^ 
x^  ■  0  and  1: 

(18)  v^3k^xk^  ■  v(3Jt|0)  (l-Xjj)  +v*3kl1*3Ck  * 

Together  (17)  and  (18)  yield  (uniquely)  the  linear  function 
required  by  (10),  and  (11)  becomes 

(19)  v (PR^  1x^*0, l£k£K1,  and  x^«  0  for  jcS)  . 

K1  . 

+  2  v(3jl)x.  +  2  (c-XA)  .x.  £  cx  , 
k=*l  K  K  jeS  33 

where  we  have  used  the  fact  that  vt^jO)  ■  0  .  it  is  clear 
from  the  discussion  following  Fig.  1  in  Sec.  3.4  that  this 


v(PR-jx]c»0  for  l£k£K^ 


/ 
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cut  is  quite  likely  to  be  violated  at  x  : 

(20)  left-hand  side  of  (19)  evaluated  at  x 

K1 

*  vlPR^Ix^Ojl^k^Kj)  +  2  ^  v(3k|l)xk 

K1 

>  v  (PR- ^-0,1^1^)  +  Z  v(3k) 

1c®  1 

=  v(PR-)  2  v(P)  ■  cx  . 

Hie  amount  of  violation  (i.e.,  the  depth  of  cut)  is  thus  at 
least  equal  to  v(PR^)  -  v(P)  ,  which  is  given  by  (9). 

Figure  1  shows  clearly  that  additional  violation  will  accrue 
for  each  x^  (l^k^l^)  which  is  fractional,  in  the  amount  by 

which  v(3k|l)xk  exceeds  vO^)  .  It  follows  .that  it  is 

best  to  take  Kx  ■  K  ,  since  this  includes  as  many  fractional 

ic^'s  as  possible. 

This  analysis  of  Example  3  may  be  summarised  by  saying 
that  a  Lagrangean  cut  can  be  obtained  for  any  subset  of  cut 
variables  which  does  not  include  any  of  the  variables  in 

u£  t  and  that  the  deepest  cuts  ars  obtained  when  the 

cut  variables  include  all  k(l£k£K)  such  that  x^  is 

fractional.  Attempts  to  include  some  of  the  variables  in 

cK  J.  as  cut  variables  will  not  always  be  successful  since 

/C*i  *» 

they  tend  to  destroy  the  essential  linearity  property  asso¬ 
ciated  with  (10) .  In  general,  then,  the  appropriate  choice 
for  the  cut  variables  is  (1,...,K)UT  .  The  corresponding 

cut  is 

K 

(21)  Z  v(3k!l)xk+Z  (c-XA)  jXj  £  cx  . 

k»l  j  €  T 


/ 


( 


33 


4.5  Further  remarks 

The  discussion  of  Examples  1-3  made  extensive  use  of  the 
fact  that  (PR^)  could  be  separated  into  a  number  of  indepen¬ 
dent  subproblems.  Such  separability  is  sufficiently  common  in 
potential  applications  of  Lagrangean  relaxation  to  warrant  the 
following  observation.  Suppose  that  the  component  indices  of 
x  can  be  partitioned  into  K  mutually  exclusive  and  exhaus¬ 
tive  subsets  (K=l  is  allowed)  such  that  (PR^) 

separates  into  K  corresponding  independent  subproblems 

v 

(Pr£) ,  k*l,  ...,K  [this  requires,  of  course,  that  none  of  the 
constraints  Bx  ^  d  link  any  of  the  subsets  of  variables]  . 


Then 


vfPR^  =  Xb  +k|1  v(PR£) 


Let  jk  be  any  particular  element  from  and  it  follows 

that 


K  v 

vfPRjJxj  ,...,Xj  )  -  Xb  4^  vtPR^IXj  ) 
Ik 


If  x.  ,  are  selected  so  that  they  are  all  0-1  vari- 

3k 

ables  then  they  constitute  a  permissible  set  of  cut  variables, 
This  is  because  [cf.  (18)]  the  linear  function 


v(p**|x.  -0)  (1-x.  )  +  v(pr£|x.  -1)x. 

JV  JJq  Jfc 


coincides  with  v(PR^|Xj  )  for  x^  ■  0  and  1  ,  and  conse¬ 


quently  (11)  becomes 


(22)  Xb  +k|1  [v(PrJ|x^-0)  (l-x^)+  v(PRj|Xj^-l)X;jJ<  CX  . 
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This  cut  has  the  property  that  it  is  always  at  least  as  deep  at  x 
as  (12) ,  the  cut  based  on  the  empty  set  of  cut  variables;  that 
is,  the  left-hand  side  of  (22)  evaluated  at  x  is  at  least  as 
large  as  v(pR^)  .  This  fact  rests  on  the  simple  observation 

v(PrJMx.  =0) (1-x.  )+  v(PR*|x.  «1)x.  £ 

Jk  Jk  -'k  ^k 

Min  (vfpR^lxj  «0),v(PR^|xj  -1))  «  v(PR*)  . 
k  k 

It  is  perhaps  unnecessary  to  remark  that  x.  need  not  be  a 

Dk 

0-1  variable  when  is  a  singleton  in  order  for  it  to  be  a 

permissible  choice  for  a  cut  variable,  for  then 

vd'R^lx.  )«(c-lA)  .  x,  itself  is  linear. 

K  3k  3k  Jk 

Two  further  remarks  are  in  order.  First,  choices  of  cut 
variables  (x.,...,x  )  which  do  not  satisfy  the  linearity  pro- 

jl 

perty  of  (10)  can  sometimes  be  accomodated  by  allowing  <  in¬ 
stead  of  equality  in  (10)  .  It  is  immediate  that  (11)  remains 
a  legitimate  cut.  Second,  (11)  can  sometimes  be  improved  by 

x  )  for  some  vectors  (x. ,...,x  ) 

n^  1  n^ 

by  a  better  lower  bound  on  v(p|x1,...,x  ).  These  two  remarks 

x  n1 

are  exploited  in  Sec.  5.5  below. 

Sec.  5.5  also  shows  how  one  may  overcome  the  apparently 
unsatisfactory  situation  for  Examples  1  and  2  in  wh&dh  there 
seems  to  be  no  way  to  construct  a  cut  of  positive  depth  at  x  . 


replacing  v(PR^  Jx^  . . . , 
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5.  RELATIONS  TO  THE  PENALTY  CONCEPT 

The  so-called  "penalty"  concept  in  integer  programming  was 
propelled  to  prominence  by  Driebeek  [ 4  ] ,  although  the  essen¬ 
tial  notion  was  used  earlier  by  Healy  [15]  and  Dakin  [  3  ]  .  The 
original  idea  was  to  underestimate  the  amount  by  which  the 
optimal  value  of  the  LP  relaxation  of  the  current  candidate 
problem  would  increase  if  separation  were  carried  out  using  a 
particular  separation  variable.  The  estimates  of  change,  re¬ 
ferred  to  as  penalties,  can  be  used  to  help  guide  separation 
and  may  aiso  permit  fathoming  or  range  restriction.  An  impor¬ 
tant  subsequent  refinement  of  this  original  idea  was  the  recogni¬ 
tion  that  computing  penalties  based  on  LP  relaxations  does  not 
come  to  grips  with  the  fact  that  it  is  the  candidate  problems 
and  subproblems  th  imselves,  and  not  their  LP  relaxations,  which 
ar fc  central  to  the  underlying  enumerative  process.  Tomlin  [19] 

[20]  showed  how  to  modify  the  penalty  formulae  so  as  to  take  at 
least  partial  account  of  the  integrality  conditions.  The  re¬ 
sulting  penalties  are  underestimates  of  the  difference  between 
v(CP)  and  the  optimal  value  of  a  candidate  subproblem  derived 
from  (CP)  .  Perhaps  needless  to  say,  in  general  it  is  not  possi¬ 
ble  to  take  complete  account  of  all  integrality  conditions  in  com¬ 
puting  these  strengthened  penalties,  for  that  would  require  just 
about  as  much  work  as  solving  (CP)  itself  .  See  [11]  for  a 
discussion  of  current  practice  in  the  computation  and  use  of 
penalties. 

Lagrangean  relaxation  furnishes  a  convenient  setting  for  de¬ 
riving  the  simple  and  strengthened  penalties  alluded  to  above. 

This  is  done  in  Sec.  5.1.  More  importantly,  it  leads  naturally 
to  extensions,  specializations,  and  additional  uses  for  penal¬ 
ties  which  do  not  follow  as  easily  from  the  more  traditional 
viewpoint.  This  is  illustrated  in  Secs.  5.2-5.  5,  the  last  of  which 
is  devoted  to  the  promising  topic  of  using  penalties  to  strengthen 
the  Lagrangean  cuts  of  Sec.  4. 
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5.1  Basic  Results:  Bx  >  d  Vacuous 
_ _ s _ _ _ 

I  The  first  task  is  to  sho«r  how  the  formulae  for  simple  and 

strengthened  penalties  are  related  to  Lagrangean  relaxation. 
This  requires  specializing  Bx  £  d  to  be  vacuous  (in  contrast 
to  our  usual  convention,  in  this  subsection  Bx  ^  d  will  not 
t  even  include  upper  bounds  on  the  variables)  .  It  follows,  as  in 

Theorem  2,  that  the  objective  function  coefficient  of  (CPR^-) 

vanishes  for  all  j  such  that  x^  is  fractional,  and  hence 
that  for  all  jelf  we  have 

VD(j)  A  v (CPRj^l x j  <  [x  j).  v(CP) 

Vy(j)  ^  v (CPR^I  ^  [Xjl  +D-  v(CP). 

» 

Thus  the  Lagrangean  relaxation  (CPRr-)  appears  to  yield  zero 
"down"  and  "up"  penalties  for  separation  on  x^  . 

A  simple  remedy  is  to  employ  an  alternative  representation 
for  Xj  in  terms  of  variables  whose  objective  function  co¬ 
efficients  in  (CPR^-)  do  not  vanish.  Such  a  representation  is 

available  from  the  final  tableau  of  the  linear  program  (CP) 
l  since  x^  must  be  basic  therein.  To  facilitate  comparison  with 

prior  work  on  penalties,  assume  for  the  remainder  of  this  sub¬ 
section  that  the  constraints  Ax  £  b  are  actually  equalities 
rather  than  inequalities  (add  slack  variables  if  necessary) . 

®  Then  x^  has  a  representation  in  terms  of  the  variables  which 

are  non-basic  (ieN)  in  the  final  LP  tableau  of  the  form 

x.«  x.-  2  • 

|  J  J  ieN  1,1  1 

The  use  of  this  representation  in  the  definition  of  Vp(j)  and 
Vy ( j)  leads  to  the  following  strengthened  conditional  bounds: 


I 


» 
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for  jel-  , 


Vg(j)  A  vCCPRj-jXj-  Z  a^ixi  <  [Xj]) 


V*(j)  =  v{CPR^-|xj-  Z^  a^ixi  >  [x.J+1). 

Clearly 

V*(j)  <  v(CP| Xj  <  [Xj]) 

V*(j)  <  v(Cp|x..  £  [Xj]+1) 

for  all  jel£  j  that  is,  these  conditional  bounds  really  do 

underestimate  the  optimal  value  of  the  candidate  problems  that 
would  result  if  (CP)  were  separated  using  x^  as  the  separa¬ 
tion  variable. 

unfortunately  the  computation  of  V*(j)  and  V*(j)  may 

be  onerous  if  a^  ^  0  for  some  variables  ieNfll  .  The  computa¬ 
tion  then  requires  solving  a  knapaach-type  problem  with  some  in¬ 
teger  variables.  Hence  it  is  natural  to  think  of  estimating 
V*(j)  and  V*(j)  from  below  by  simply  dropping  all  integrality 
conditions: 


vD°(j)  4  v(CPRrl*j-.2„  “jixi  s  t*ji) 


V*°(j)  A  vtCPRj-lxj-  Z  a^ixi  ^  [Xjl+D 


The  computation  of  each  of  these  conditional  bounds  merely  re¬ 
quires  minimizing  a  linear  function  with  all  non-negative  coeffi* 
cients  [  (c-Ta)  0  by  duality]  subject  to  a  single  linear  con¬ 

straint  and  x  £  0  .  This  is  sometimes  referred  to  as  a 
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"continuous  knapsack"  type  problem  and  it  is  easy  to  write  down 
an  explicit  solution: 

v!°(j)  »  V(CP)+(X.-  [X  ])  Miniatttm  f  (6-XX)  . 

3  3  i€N:  ;  — - i 


ji 


>0  aji 


(25) 


V*° ( j)  =  v(CP) +( [x.]+l-x.)  Minimum  f  (c-Ta)  . 
U  j  J  i  CM.  J  - i 


leN: 

»n<°  /•■"ji  > 

(we  have  used  the  fact  that  Xb«v  (CP)  by  LP  duality)  .  These 
conditional  bounds  are  identical  to  those  associated  with  the 
simplest  penalties  mentioned  earlier  (cf.  (5).  in  [4  ]). 

The  strengthened  penalties  of  Tomlin  can  also  be  recovered 
from  this  viewpoint  by  retaining  the  condition  that  x^  must 
not  be  in  the  open  interval  (0,1)  for  jeNfll.  Then  we  obtain 

1  /  _£  % 


Vnl  j)*v(cPRr-|x,-  2  a.  .x,  <  [x.]  and  x./(0,l)  for  all  ieNfll) 
D  k  3  ieN  31  1  “  3  1 


(26) 


V*X( j)^v(CPRr-|x.-  2  a.  .x.  j>  [x.J+1  and  x./(0,l)  for  all  ieNfll) 
U  A  J  i  cV  J 1  1  J  1 


ieN 


It  is  not  difficult  to  see  that  at  most  one  variable  need  be  at 
a  positive  level  in  an  optimal  solution  of  the  modified  continuous 
knapsack  problems  defined  in  (26)  .  This  observation  leads  to  the 
explicit  formulae: 

V^jJ-vfCPj-tMin  /(c-JTa)  if  i/l 


(27) 


i>0  c-rA)  AMax  ^(Xj-fx^J)  ^ 


if  i€l 


Vy  (j)-v(CP)4Min  Mc-rA)  i([xj]+l-xj)/(-aji)  if  i/I 
aiji<0  I  (c-^A)  ^ax  Gx^J+l-x^  ^  if  iel 


(-»ji)  j 
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These  formulae  are  identical  with  the  strengthened  penalties  of 
Tomlin  (cf.  (10)  and  (11)  of  [19]  or  (3.5)  and  (3.6)  of  [20]). 

It  is  evident  from  the  very  definitions  (23),  (24)  and  (26)  that 

V*°(j)  <vd1(^ 

(28)  for  all  jel£  . 

V*°(j)  !V*X(j)  <V*(j) 

This  completes  the  recovery  of  known  formulae  for  Driebeek 
and  Tomlin  penalties  for  jcl£  . 

Exactly  the  same  type  of* analysis  holds  for  penalties  associated 
with  basic  variables  in  I-If.  Such  penalties  are  of  interest 
as  a  me  am  s  of  obtaining  tighter  ranges  on  integer  variables  which 
happen  to  be  naturally  integer  in  the  optimal  solution  of  (CP)  . 

For  a  nonbasic  variable  in  I-If  the  quantity  of  interest 

is  v  (CPR^-|  x j  J>  1) ;  no  alternative  representation  in  terms  of 

nonbasic  variables  is  possible.  Evidently 

(29)  v (CPRj£-|  x j  ^  l)-v(CP) +(c-^A)  j  . 

Again  this  is  a  standard  result. 

Another  technique  for  strengthening  (24)  makes  use  of  the 
following  elementary  result. 

Theorem  3. 

Let  (IP)  be  a  minimising  integer  linear  program  in  which 
exactly  one  variable,  say  x^  ,  is  declared  to  be  in¬ 
teger-valued.  Suppose  that  x^  ,  the  optimal  level 
of  x^  when  (IP)  is  solved  ignoring  the  integrality 
requirement,  is  fractional.  Then  the  optimal  value 
of  (IP)  is  given  by 

v (IP) -Min  (vfXPlx^Ixjj)  ^(IpIxj-fx^J+l)  ) . 
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Proof .  We  have  assumed  (without  loss  of  generality) that  (IP)  is  a  mini¬ 
mizing  problem.  Then  it  is  well-known  that  v(ip|x^)  is  a  con¬ 
vex  function  of  with  minimal  value  at  x^  .  The  desired 
conclusion  follows  easily.  The  possibility  that  (ip| x^» [x^] ) 

or  (Tp|  x^*  [x^]  + 1)  or  both  are  infeasible  is  not  excluded 
(recall  that  our  convention  is  to  define  a  minimum  over  an  empty 

set  aS  +'>'  Q.E.D. 

Let  iD(j)  be  the  minimizing  non-basic  i  in  the  formula 

for  V*°(j)  given  in  (25)  .  The  index  iy(j)  is  defined  simi¬ 
larly.  Then  application  of  Theorem  3  in  the  obvious  way  permits 
the  following  improvement  on  (24)  to  be  computed  with  only  a 
little  extra  effort: 


I 


VD2  <  j )  “v  (CPRX"I x j “ . 2  *jixi  £  l*j]  and  xi  (j)  int®9ttr> 


ieN 


(30) 


V*2(j)=v(cPR^|xj-  2  8jixi  ^  txjj+i  and  xL  integer). 


ieN 


Neither  (30)  nar  (26)  necessarily  dominates  the  other.  It  is 
not  difficult  to  see  that  the  following  relationship  then  holds 
for  jelf: 


*1 
t  *  i 


<xj-'xjl,/  ajin(J)/s  \l  *  MVJ2(J) 


(31) 


([xjl+l-xj)/(-ajiu<j))  JH  1  -  V^j)  r(v*2( j). 


We  are  unable  to  supply  a  reference  to  the  conditional  bounds 
(30)  in  the  published  literature. 


j 

I 


/ 


So  far  we  have  required  Bx  £  d  to  be  vacuous ,  that  is, 
all  upper  bounds  and  other  special  constraints  are  treated  as 
general  A-type  constraints.  Analogs  of  the  previous  penalty  re¬ 
sults  as  well  as  new  penalty  results  emerge  easily  by  allowing 
Bx  £  d  to  be  non-vacuous.  This  will  now  be  illustrated  for  the 
three  examples. 

The  requirement  made  throughout  this  subsection  that 
Ax  >  b  be  replaced  by  Ax  »  b  in  (P)  served  purely  as  a 
means  of  simplifying  notation  and  facilitating  comparison  with 
equivalent  results  in  the  previously  published  literature. 

This  requirement  is  now  dropped. 
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5.2  Penalties  for  Example  1 

Example  1  differs  from  the  previous  development  only 
in  that  (CPR^)  now  has  upper-bounded  variables.  Both 
VD(j)  and  Vy(j)  still  equal  v(CP)  for  all  j  e  I£  by  Theorem 
2.  The  remedy  for  these  vanishing  penalties  is  again  to 
invoke  the  representation  for  x^  which  is  available  from  the 
final  LP  tableau  of  (CP) .  This  representation  will  be 
written  as 


(32) 


[j  "  *jo 


-  2 


or*  4  Xj 


ji 


for  j  e  If 


where,  of  course,  many  of  the  coefficients  cr^  may  be  0. 

The  resulting  strengthened  conditional  lower  bounds  on 
v (CP Jxj  £  (Xj])  and  v(CP(x^  ^  Sj  ]  +  D  for  j  €  If  are 

vd<J)  *  v(CPRrix3  •  «jo xi  ^ 

,33)  .  4  _ 

vD(j)  -  vttPRj-lxj  -  Bjo  -  2  aji  **  2  ffjl  +  1). 

*  * 

We  have  used  the  notation  VQ  and  Vy  as  in  (23)  because  (33) 
is  an  exact  counterpart  of  (23).  Like  (23),  (33)  is  generally 
computationally  onerous  because  each  estimate  requires  solving 
a  knapsack-type  problem  in  integer  variables.  The  fact  that 
the  knapsack  problem  now  has  upper-bounded  variables  is  but 
a  small  advantage.  The  most  easily  computed  lower  approxi¬ 
mation  to  (33)  is  obtained  by  dropping  the  integrality  re¬ 
quirements  as  in  (24) : 

V*°(j)  -  v(CP^.|Xj  -  0fjo  -^erji  x±  £  Cxjl) 

V*°(j)  «  v(CPR^|Xj  -  orjo  Xi  ±  +  1) - 


(34) 
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The  notation  V*°  and  V*°  has  again  been  carried  over. 

The  differences 

(35)  V*°(j)  "  v(CP)  and  V*°(j)  -  v(CP) 

are  Driebeek-like  up  and  down  penalties  for  Example  1. 

The  computation  of  (34)  requires  only  slightly  more  effort 
than  the  computation  of  (24).  A  "continuous  knapsack" 
problem  with  upper-bounded  variables  must  now  be  solved. 
Explicit  formulae  for  V*°  and  Vg°  are  slightly  more  cumber¬ 
some  than  expression  (25),  but  are  easily  programmed  for  a 
digital  computer. 

To  strengthen  (34)  one  may  formally  write  the  counter¬ 
part  of  (26),  but  unfortunately  explicit  calculation  may  be 
nearly  as  costly  as  that  of  (33)  itself.  This,  is  because 
the  upper  bounds  generally  invalidate  the  key  property  of 
(26)  that  at  most  one  variable  need  be  at  a  positive  level 
in  an  optimal  solution  of  each  associated  optimization 
problem.  Thus  the  strengthened  penalties  of  Tomlin  do  not 
generalize  usefully  to  Bx  d  when  it  includes  upper  bounds 
on  variables. 

But  the  other  technique  based  on  Theorem  3  for  strength¬ 
ening  VD°(j)  and  V^°(j)  does  generalize  nicely.  Let  ijj(j) 
and  iTT(j)  be  respectively  the  fractional-valued  variables 
in  the  solutions  of  the  optimizations  corresponding  to  V_  (j) 
and  V(J  (j).  It  is  easy  to  see  that  at  most  one  variable  need 
be  fractional  in  each  of  these  solutions  t  if  none  is,  then 
that  penalty  cannot  be  strengthened  by  the  present  device. 

The  strengthened  conditional  bounds  analagous  to  (30)  ares 


f 
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VD2(j)  »  v^PR^JXj  -  tfj0  -^2  aji  xA  £  fx^  and  x.^(j)  Integer) 


(36) 


vj2  ( j)  "  V 


(CPRX  l*j  "  “jo  _  *  «Ji  xi  ^  and  xiD(j)  inte9er*' 


The  required  optimizations  are  inexpensive  to  carry  out  on  a 
digital  computer.  Clear ly, 


(37) 


v^°(j)  i  v*2(j)  £  v*(j)  £  vWPlXj  i  UjU 
V*°(j)  £  V*2(j)  i  v*(j)  i  vlcp^  i  iSjl  +  1). 


Exactly  the  same  types  of  penalties  can  be  constructed  for 
j  €  i-if  when  the  objective  function  coefficient  of  x^  vanishes 
in  (CPR^j . 

5.3  Penalties  for  Example  2 

The  development  of  penalties  for  Example  2  closely  par¬ 
allels  that  for  Example  1.  For  j  €  If  both  up  and  down  pen¬ 
alties  again  vanish  by  Theorem  2,  and  it  is  necessary  to  use 
the  final  LP  tableau  representation  of  the  form  (32)  The 
resulting  conditional  bounds  VD(j)  and  V^(j)  defined  in  (33) 
may  still  be  too  computationally  onerous  to  use  in  general, 
but  the  multiple  choice  constraints  do  tend  to  make  the  com¬ 
putation  easier  by  comparison  with  Example  1.  One  can  imagine 
nontrivial  situations  where  VD(j)  and  Vy(j)  can  be  computed 
relatively  economically  by  a  simple  enumeratlve  procedure. 

But  in  general  one  may  have  to  fall  i>ack  on  the  Driebeek-like 
penalties  defined  by  (34)  .  The  required  computations  are 

no  longer  simple  continuous  knapsack  problems  with  upper- 
bounded  variables,  but  they  can  still  be  carried  out  efficiently 


‘Numbered  displays  from  the  discussion  of  Example  1  Jill  be 
used  here  with  the  understanding  that  (CP),  etc.,  have  ihe 
structure  of  Example  2  rather  than  Example  1. 


by  specialized  techniques  (e.g.,  by  parametric  optimization 
applied  to  the  dual  of  (CPR^-)  with  respect  to  the  added  con¬ 
straint)  .  Strengthening  these  penalties  along  the  lines 
suggested  by  Tomlin  as  in  (26)  appears  to  be  no  easier  in  general 
than  (33)  itself.  But  again,  as  with  Example  1,  the  strength¬ 
ening  (36)  based  on  Theorem  3  is  attractive.  The  indices 
iQ(j)  ai.d  iy(j)  may  be  selected  to  be  any  of  the  fractional¬ 
valued  variables  in  the  solutions  of  the  optimizations  cor¬ 
responding  to  V*°(j)  and  V*°(j).  The  implementation  of  (36) 
on  a  digital  computer  is  only  slightly  more  expensive  them 
that  of  (34).  Naturally,  (37)  continues  to  hold.  The  reader 
should  have  no  difficulty  seeing  what  to  do  if  penalties  are 
desired  for  variables  in  I-I^  for  reasons  explained  in  Section  3.5. 

The  special  nature  of  the  multiple  choice  constraints 
(1)  makes  it  possible  to  define  "cumulative"  conditional  upper 
bounds  as  follows: 

VU°(j;Jk)  *  n®*  V*°(i)  for  1  €  {Jk~j}} 

(38)  *  A 

vU2(j*Jk)  *  Max  {v*u2(j),  V*2  (i)  for  i  c  {jk"j}} 

where  it  is  understood  that  j  is  in  Jk  in  these  definitions. 

That  this  provides  true  lower  bounds  on  v  (CP  |x^  -  1)  follows 
from  the  fact  that  x^  ■  1  implies  x^  ■  0  for  all  ij*j  in  the 
same  multiple  choice  set. 

5.4  Penalties  for  Example  3 

For  Example  3  we  must  partition  into  two  parts: 

If  n  {i,...,k}  and  lf-^f  fl  {l, ...,Kp.  For  the  first  part, 
as  described  in  detail  in  Section  3.4,  the  up  and  down  pen¬ 
alties  associated  with  VD(j)  and  Vy(j)  are  highly  unlikely 
to  vanish.  In  fact,  they  are  likely  to  be  quite  large.  Thus 
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it  is  unnecessary  to  invoke  an  alternative  representation 
for  such  j  from  the  final  LP  tableau.  Por  j  £  If"^f  n 
however,  it  is  easy  to  see  that  the  naive  penalties  varnish  and' 
thus  that  the  alternative  representation  may  be  useful.  Die 
detailed  discussion  would  be  so  close  to  that  for  Example  2 
that  it  will  not  be  given  here. 

5.5  Penalties  and  Lagrange an  cuts 

We  have  seen  that  the  Lagrangean  relaxation  viewpoint  is  con¬ 
sistent  with  standard  penalty  results,  and  that  various  improvements 
thereon  for  special  problem  structures  such  as  Examples  1-3  then 
follow  easily.  What  to  do  with  these  penalties  has  not  been 
discussed.  For  this  the  reader  should  refer  to  Sections  3.4 
and  3.5;  the  uses  described  there  include  all  of  the  standard 
uses  for  penalties  to  be  found  in  the  published  literature. 

An  additional  use  for  penalties  is  in  connection  with 
Lagrangean  cuts,  via  the  options  described  in  Section  4.5. 

Recall  for  Examples  1  and  2  that  all  cuts  of  type  (11)  have 
zero  depth  at  x  no  matter  which  cut  variables  are  selected. 

Both  the  simple  penalties  based  on  (34)  and  the  strengthened 
penalties  based  on  (36)  can  be  used  to  generate  cuts  with 
positive  depth  at  x  so  long  as  at  least  one  of  these  penalties 
is  non-zero.  Ibis  may  be  done  as  follows.  In  order  to  keep 
the  notation  below  consistent  with  that  of  Section  4,  it  will 
be  assumed  without  loss  of  generality  that  (CP)  equals  (P) 
when  applying  the  results  of  Sections  5. 2-5. 4. 

Consider  first  the  simple  conditional  bounds  (34) . 

Select  any  j  t  If  such  that  at  least  one  of  the  penalties 
(35)  is  strictly  positive,  and  take  this  j  to  be  the  one 
cut  variable .  Clearly, 


(39)  v (PR^|Xj  -  <*j0  ".^®ji  xi*  ^  v(pl*j> 

holds  for  all  fixed  Xj,  and  in  particular  for  all  values  of 
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Xj  which  are  part  of  a  feasible  solution  to  (P)  •  The  left- 
hand  side  of  (39)  is  convex  as  a  function  of  x^,  and  thus 
the  unique  linear  function  passing  through  it  at  the  points 
fxj*|  and  fxjl  +  1  must  underestimate  it  for  all  integer  values 

Of  XjS 

(40)  V*°(j)  +  (V*°(j)  -V*°(j>)  (Xj  -  £ 


vfpR^JXj 


-  2  a**  x.)  for  all  integer  x.  . 

JO  ji  i  J 


Together  (39)  and  (40)  imply  that 


(41)  V*°(j)  +  (V*°(j)  -  V*°(j)>  <Xj  -  fr,])  £  cx 


is  a  legitimate  cut.  This  cut  can  be  thought  of  as  a  rela¬ 
tive  of  (11)  which  takes  advantage  of  the  two  remarks  in  the 
closing  paragraph  of  Section  4.5.  Notice  that  if  there  are 
several  j  c  If  for  which  VD°(j)  and  vjy  (j)  are  computed,  then 
it  is  an  easy  matter  to  select  j  so  as  to  yield  the  cut  of 
type  (41)  which  is  deepest  at  x. 

Now  consider  the  strengthened  ;  jnditional  bounds  (36). 

An  analog  of  inequality  (39)  holds,  but  the  analog  of  (40) 
does  not  because  of  the  added  integrality  requirement  in  (36) . 
It  appears  necessary  to  require  that  j  e  be  a  0-1  variable 
if  a  cut  is  to  be  based  on  (36)  with  j  as  the  single  cut 
variable.  Ihen 


(42)  VD<6(j)  +  (V^(j)  -  VD‘(j))  x^cx 

is  a  legitimate  cut  related  to  the  original  cut  (11)  by  the 
last  nemark  of  Section  4.5.  By  (37),  (42)  is  clearly  a 
superior  cut  to  (41)  .It  is  a  simple  matter  to  select  j  so  as 
to  yield  the  cut  which  is  deepest  at  x  among  those  of  the  form 


T - - 


For  Example  2  one  should  of  course  use  in  (41)  and  (42) 

the  cumulative  penalties  defined  in  (38)  in  place  of  Vn°(j) 

*2  u 
or  Vy  ( j )  if  the  necessary  quantities  are  at  hand.  One  may 

further  improve  cuts  (41)  and  (42)  when  j  is  a  multiple 

choice  variable  by  using  one  of  the  obvious  cuts 

(43)  2  V*°(j?Jk)  x.  £  cx,  k«l,...,K 

jcjk  D  3 

or  the  still  stronger  cuts 

(44)  2  V*2(j?Jv)  x.  £  cx,  k-l,...K. 

j£jk  0  J 

Each  of  these  cuts  takes  all  of  as  the  set  of  cut  variables, 
and  is  related  to  (11)  by  the  last  remark  of  Section  4.5.  It 
is  easy  to  /erify  that  cuts  of  the  form  (43)  [resp.  (44)  ]  are 
at  least  as  strong  as  those  of  the  form  (41)  [resp.  (42)]  for 
all  x  feasible  in  (p) . 

A  cut  very  much  in  the  spirit  of  (43)and  (44)  was  pro¬ 
posed  by  Healy  [15],  To  be  precise,  for  the  k*”*1  cut  he  used 

VIT°(j)  as  the  coefficient  of  x.,  where  Vu°(j)  is  computed 

u  J  u  th 

with  Bx  £  d  taken  to  consist  of  only  the  k  multiple  choice 

constraint  (no  upper  bounds  or  other  multiple  choice  constraints 
are  included) .  This  cut  is  legitimate  for  the  same  reason 
that  (43)  is,  and  is  dominated  by  (43)  because  the  coefficients 
of  (43)  are  at  least  as  large. 

We  note  in  closing  that  penalty-based  cuts  with  more 
than  one  cut  variable  can  often  be  obtained  for  Examples  1 
and  2  and  other  structures  by:  (i)  adding  to  (PR^)  relations 
of  the  form  (32)  for  any  subset  of  j* a  in  so  long  as  no  vari¬ 
able  appears  with  a  nonzero  coefficient  in  more  than  one  of 


these  relations ,  and  then  (ii)  exploiting  separability. 

See  the  initial  discussion  of  Section  4.5  for  a  development 
in  much  this  same  spirit.  Details  are  left  to  the  reader. 
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6.  RELATIONS  TO  SURROGATE  CONSTRAINTS 


The  surrogate  constraint  is  a  device  which  was  originally 
developed  in  the  context  of  pure  0-1  integer  programs.  There 
is  nothing  inherent  in  the  nature  of  this  device  which  limits 
its  use  to  this  context ,  as  will  become  amply  clear,  but  it 
will  be  more  convenient  to  conduct  our  brief  review  of  surrogate 
constraint  ideas  assuming  until  further  notice  that  (P) 
is  a  purely  0-1  program  with  Bx  ^  d  consisting  solely  of 
the  unit  upper  bounds. 

Glover  [12  ]  proposed  that  surrogate  constraints  of  the  form 
X  (b-Ax)  <0  with  fixed  X  0  be  added  to  a  0-1  integer  pro¬ 
gram  in  conjunction  with  its  solution  by  an  enumerative  algorithm. 
Any  X  >  0  leads  to  a  surrogate  constraint  which  is  legiti¬ 
mate  in  the  sense  that  it  must  be  satisfied  by  all  x  feasible 
in  (P)  ,  for  it  is  merely  a  nonnegative  linear  combination  of 
the  given  constraints.  Such  constraints  were  conceived  pri¬ 
marily  for  use  with  enumerative  algorithms  that  do  not  use  an 
LP  subroutine,  for  such  algorithms  have  very  limited  ability 
to  consider  the  conjoint  implications  of  several  constraints. 
Glover's  criterion  for  the  choice  of  X  was  that  it  should 
maximize 

Minimum  cx  subject  to  X (b-Ax)  <  0 
x-0, 1 

over  all  X  £  0  .  unfortunately,  this  criterion  is  prohibitively 
expensive  to  implement  in  general.  By  dropping  the  integrality 
requirements  while  keeping  0  £  x  £  1  ,  however,  Balas  [  1  ] 
showed  that  the  modified  criterion  permitted  a  characterization 
of  the  optimal  X  as  the  dual  solution  T  corresponding  to 
the  constraints  Ax  >  b  in  the  ordinary  linear  program  (p)  . 

This  leads  to  a  surrogate  constraint  that  is  identical  to(13)withX»r, 
which  we  remarked  earlier  is  identical  to  Balas*  "filter" 
constraint . 


>v* 


*■  Y'Wn^O-T' 
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The  present  author  took  a  different  route  in  [ 8 ]  by  adding 
^  the  constraint  cx  <  z  to  X  (b-Ax)  £0  [recall  from  Sec.  3.1 

that  z  is  the  objective  function  value  associated  with  the 
best  known  feasible  solution  to  (P)]  and  rationalizing  the 
following  slightly  different  criterion  for  the  choice  of  X  : 
maximize 

l  * 

Minimum  X (b-Ax)  4 (cx-z  ) 

x»0, 1 

over  X  ^  0  .  This  criterion  also  leads  to  JT  as  the  optimal 
II  choice  for  X  .  The  resulting  surrogate  constraint  advocated  in 

[  8  ]  is 

(45)  cx+T(b-Ax)  <  z*  . 

I 

Two  uses  of  (45)  were  proposed:  as  a  possible  means  of  fathoming 
via  the  easy  test 

r  ~|  ?  ^ 

.  (46)  Minimum  cx+r(h-Ax)  z 

9  L  x*=0, 1  J 

and  as  a  possible  means  of  range  restriction  via  the  following 
easy  tests  applied  to  the  jth  variable: 

; 

(47a)  f  Minimum  cx+Mb-Ax)  j  >  z* 
x*0,l  s.t.  Xj«0 J 


(47b)  [  Minimum  cx+r (b-Ax)  1  z*  . 

L  x«0,l  s.t.  Xj«lJ 

If  (46)  holds  then  (P)  is  fathomed.  If  (47a)  [reap.  (47b)] 
holds  then  x^  must  be  1  [resp.  0]  in  any  feasible  solution  of 
(P)  which  is  superior  in  value  to  the  current  incumbent .  It  is 
important  to  add  that  not  only  would  these  tests  be  performed 
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as  soon  as  (45)  is  created,  but  also  subsequently  for  candidate 
problems  derived  from  (P)  by  separation  --  in  which  case  the 
accumulated  separation  constraints  would  be  appended  to  the 
optimizations  in  (46)  and  (47)  .  Note  that  this  does  not  spoil 
the  essential  triviality  of  (46)  and  (47)  from  a  computational 
standpoint . 

It  is  easy  to  interpret  (46)  and  (47)  from  the  viewpoint 
of  Lagrangean  relaxation  with  Bx  £  d  consisting  of  the  upper 
bound  constraints  x..  <  1  .  Test  (46)  can  be  rewritten  as 

?  * 

(48)  vtPR^)  >  z  , 

which  is  precisely  the  elementary  fathoming  criterion  associated 
with  (PR^-) •  Similarly,  (47)  can  be  rewritten  as 

?  * 

(49a)  v  (PR^-|  Xj*0)  £  z 

(49b)  v (PR^-|  Xj**l)  £  z*  . 

This  is  precisely  the  range  restriction  criterion  described  in 
Sec.  3.5.  When  (46)  and  (47)  are  performed  on  subsequent  candi¬ 
date  problems  derived  from  (P)  by  separation,  (48)  and  (49)  are 
still  standard  Lagrangean  relaxation  tests  except  that  an  "old" 

X  is  used  in  (CPR^) .  This  is  the  standard  "pre-LP"  strategy 
mentioned  in  Sec.  3.7. 

Thus  the  surrogate  constraint  (45)  and  the  tests  based  on 
it  are  seen  to  be  completely  subsumed  by  the  simplest  Lagrangean 
relaxation  techniques.  Generalizations  of  (45)  »■  (47)  when 
(P)  is  not  a  pure  0-1  program  or  when  Bx  ^  d  includes  more 
than  simple  upper  bounds  can  be  obtained  without  difficulty. 

Some  such  generalizations  were  developed  a  few  years  ago  by  this 
writer  in  unpublished  lecture  notes  and  by  Glover  [13]  using  the 
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surrogate  constraint  viewpoint ,  but  in  every  case  the  same 
results  are  easy  special  cases  of  more  general  and  powerful 
results  based  on  Lagrangean  relaxation.  It  therefore  appears 
that  the  surrogate  constraint  viewpoint  can  be  discarded  in 
favor  of  the  Lagrangean  relaxation  viewpoint. 

I 


I 


I 


I 


I 


» 
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7.  CONCLUSION 

ft  We  hope  that  the  length  of  this  development  has  not 

obscured  the  basic  ideas  that  we  have  tried  to  convey. 
Fore.aost  among  these  is  the  usefulness  of  Lagrangean  relax¬ 
ation  as  a  unified  viewpoint  from  which  one  may  develop 
I  auxiliary  devices  to  improve  the  performance  of  existing 

branch-and-bound  algorithms  for  integer  programming  (Sec.  3) , 
develop  new  cutting-planes  (Sec.  4) ,  develop  new  penalties 
(Sec.  5),  and  supplant  the  narrower  notions  of  surrogate 
I  constraints  (Sec.  6)  and  relaxation  by  simply  discarding 

troublesome  constraints.  All  of  these  things  usually  can 
be  done  in  a  way  that  exploits  the  special  structure  of  the 
particular  problem  class  at  hand.  The  trick  is  to  judiciously 
l  choose  the  subset  of  constraints  to  play  the  role  of  Bx  ^  d 

in  our  statement  of  the  general  problem  (P) .  The  three  examples 
of  Sec.  1.1  were  carried  throughout  the  paper  to  indicate  how 
flexible  the  application  of  Lagrangean  relaxation  can  be,  and 
l  how  dependent  the  results  are  on  the  particular  choice  of 

Bx  ^  d.  The  integrality  Property  first  encountered  in  Theorem 
1  appears  to  yield  a  useful  dichotomy  of  the  universe  of  pos¬ 
sible  choices  for  Bx  ^  d;  choices  satisfying  this  property 
t  seem  to  have  less  inherent  potential  than  choices  not  satis¬ 

fying  it,  and  often  require  extra  measures  to  achieve  attrac¬ 
tiveness  (this  is  the  main  subject  of  Sec.  5). 

(  There  is  much  work  yet  to  be  done  to  develop  the  full  poten¬ 

tial  usefulness  of  Lagrangean  relaxation  for  various  special 
problem  classes.  Several  basic  strategies  require  further  ex¬ 
ploration,  as  discussed  in  Sec.  3.7.  The  recent  work  in  this 
vein  cited  earlier  [5]  [6]  [7]  [16]  [17]  is  an  auspicious  be¬ 
ginning,  but  is  still  just  a  beginning. 
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To  this  list  can  be  added  the  current  work  of  the  author  [10]  on 
warehouse  location  problems  with  lower  as  well  as  upper  capacities 
on  warehouse  throughput,  and  with  arbitrary  additional  constraints. 
Such  problems  require  but  a  slight  generalization  of  Example  3  in 
this  paper.  The  algorithm  of  [8]  (see  also  Sec.  3.1.5  of  [11])  has 
been  modified  to  use  both  cuts  and  penalties  based  on  Lagrangean 
relaxation.  Preliminary  computational  experience  vith  several 
practical  problems  suggests  that  the  first  Lagrangean  cut  is  much 
more  effective  than  Gomory's  1960  mixed-integer  cut;  and  that  the 
Lagrangean  penalties  are  typically  at  least  an  order  of  magnitude 
higher  than  Tomlin's  strengthened  penalties,  yet  less  expensive 
to  compute.  The  net  result  is  that  overall  computing  times  are 
greatly  reduced,  we  feel  that  this  experience  alone  justifies 
further  serious  investigation  for  other  significant  classes  of 
problems . 
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